JudgeLM: Fine-tuned Large Language Models are Scalable Judges

Evaluating Large Language Models (LLMs) in open-ended scenarios is challenging because existing benchmarks and metrics can not measure them comprehensively. To address this problem, we propose to fine-tune LLMs as scalable judges (JudgeLM) to evaluate LLMs efficiently and effectively in open-ended benchmarks. We first propose a comprehensive, large-scale, high-quality dataset containing task seeds, LLMs-generated answers, and GPT-4-generated judgments for fine-tuning high-performance judges, as well as a new benchmark for evaluating the judges. We train JudgeLM at different scales from 7B, 13B, to 33B parameters, and conduct a systematic analysis of its capabilities and behaviors. We then analyze the key biases in fine-tuning LLM as a judge and consider them as position bias, knowledge bias, and format bias. To address these issues, JudgeLM introduces a bag of techniques including swap augmentation, reference support, and reference drop, which clearly enhance the judge's performance. JudgeLM obtains the state-of-the-art judge performance on both the existing PandaLM benchmark and our proposed new benchmark. Our JudgeLM is efficient and the JudgeLM-7B only needs 3 minutes to judge 5K samples with 8 A100 GPUs. JudgeLM obtains high agreement with the teacher judge, achieving an agreement exceeding 90% that even surpasses human-to-human agreement. JudgeLM also demonstrates extended capabilities in being judges of the single answer, multimodal models, multiple answers, and multi-turn chat.Comment: 30 pages, 23 figure

WeakTr: Exploring Plain Vision Transformer for Weakly-supervised Semantic Segmentation

This paper explores the properties of the plain Vision Transformer (ViT) for Weakly-supervised Semantic Segmentation (WSSS). The class activation map (CAM) is of critical importance for understanding a classification network and launching WSSS. We observe that different attention heads of ViT focus on different image areas. Thus a novel weight-based method is proposed to end-to-end estimate the importance of attention heads, while the self-attention maps are adaptively fused for high-quality CAM results that tend to have more complete objects. Besides, we propose a ViT-based gradient clipping decoder for online retraining with the CAM results to complete the WSSS task. We name this plain Transformer-based Weakly-supervised learning framework WeakTr. It achieves the state-of-the-art WSSS performance on standard benchmarks, i.e., 78.4% mIoU on the val set of PASCAL VOC 2012 and 50.3% mIoU on the val set of COCO 2014. Code is available at https://github.com/hustvl/WeakTr.Comment: 20 pages, 11 figure

A Single Mode 980nm InGaAs/GaAs/AlGaAs Large Optical Cavity Quantum Well Laser with Low Vertical Divergence Angle

To achieve high optical power as well as low vertical divergence angle, a new kind of optimized large optical cavity (LOC) structure is applied to a ridge waveguide 980nm InGaAs/GaAs/AlGaAs multi-quantum well laser. The optical power density in the waveguide is successfully reduced. The maximum output power is more than 400mW with a slope efficiency of 0.89W/A and the far-field vertical divergence angle is lowered to 23°

Enhancing ultra-wideband THz fingerprint sensing of unpatterned 2D carbon-based nanomaterials

THz molecular fingerprint sensing is a promising non-destructive method to accurately detect ultra-thin carbon-based materials in the nanoscale. Due to their extremely low THz absorption, plasmonic metamaterials or all-dielectric metasurfaces have been adopted to enhance the light-matter interaction for detection. However, they cause considerable parasitic losses or complicated material processing on a patterned surface. Here, we propose a lithography-free all-dielectric sensor to enhance THz absorption via an evanescent wave, which can lead to high detecting performance by a coupled mode. In view of the molecular broadband features, we use a thickness-multiplexed scheme to boost the detection of fingerprint significantly. The enhancing factor for the minimum fluctuation of fingerprint feature points is up to 534. Our method drastically enhances the broadband fingerprint intensity of the ultra-thin nanoscale layer and make it comparable to that of a 700-times thick sample layer, measured with a regular approach. Our study paves the way for broadband THz fingerprint sensing of trace-amount analytes and will inspire many burgeoning THz detection applications on 2D or ultra-thin carbon-based nanomaterials.</p

Metalorganic Chemical Vapor Deposition of GaNAs Alloy Using Dimethylhydrazine as Nitrogen Precursor

GaNAs alloy is grown by metalorganic chemical vapor deposition (MOCVD) using dimethylhydrazine (DMHy) as the nitrogen precursor. High-resolution X-ray diffraction (HRXRD) and secondary ion mass spectrometry (SIMS) are combined in determining the nitrogen contents in the samples. Room temperature photoluminescence (RTPL) measurement is also used in characterizing. The influence of different Ga precursors on GaNAs quality is investigated. Samples grown with triethylgallium (TEGa) have better qualities and less impurity contamination than those with trimethylgallium (TMGa). Nitrogen content of 5.688% is achieved with TEGa. The peak wavelength in RTPL measurement is measured to be 1278.5nm

High power AlGaInP laser diodes with zinc-diffused window mirror structure

The technology of zinc-diffusion to improve catastrophic optical damage (COD) threshold of compressively strained GaInP/AlGaInP quantum well laser diodes has been introduced. After zinc-diffusion, about 20-μm-long region at each facet of laser diode has been formed to serve as the window of the lasing light. As a result, the COD threshold has been significantly improved due to the enlargement of bandgap by the zinc-diffusion induced quantum well intermixing, compared with that of the conventional non-window structure. 40-mW continuous wave output power with the fundamental transverse mode has been realized under room temperature for the 3.5-μm-wide ridge waveguide diode. The operation current is 84 mA and the slope efficiency is 0.74 W/A at 40 mW. The lasing wavelength is 656 nm

Impaired learning and memory generated by hyperthyroidism is rescued by restoration of AMPA and NMDA receptors function

Hyperthyroidism has been identified as a risk factor for cognitive disorders. The hippocampus is a key brain region associated with cognitive function, among which excitatory synapse transmission plays an important role in the process of learning and memory. However, the mechanism by which hyperthyroidism leads to cognitive dysfunction through a synaptic mechanism remains unknown. We investigated the synaptic mechanisms in the effects of hyperthyroidism in an animal model that involved repeated injection of triiodothyronine (T3). These mice displayed impaired learning and memory in the Novel object recognition test, Y-maze test, and Morris Water Maze test, as well as elevated anxiety in the elevated plus maze. Mature dendritic spines in the hippocampal CA1 region of hyperthyroid mice were significantly decreased, accompanied by decreased level of AMPA- and NMDA-type glutamate receptors in the hippocampus. In primary cultured hippocampal neurons, levels of AMPA- and NMDA-type glutamate receptors also decreased and whole-cell patch-clamp recording revealed that excitatory synaptic function was obviously attenuated after T3 treatment. Notably, pharmacological activation of AMPAR or NMDAR by intraperitoneal injection of CX546, an AMPAR agonist, or NMDA, an NMDAR agonist can restore excitatory synaptic function and corrected impaired learning and memory deficit in hyperthyroid mice. Together, our findings uncovered a previously unrecognized AMPAR and NMDAR-dependent mechanism involved in regulating hippocampal excitatory synaptic transmission and learning and memory disorders in hyperthyroidism

Maternal Vitamin D Status and Risk of Gestational Diabetes: a Meta-Analysis

Background/Aims: Whether maternal vitamin D deficiency is associated with gestational diabetes remains controversial. This meta-analysis aimed to systematically evaluate published evidence on the association between maternal vitamin D status and the risk of gestational diabetes. Methods: We retrieved relevant articles from the PubMed, Medline and Embase databases up to May 2017 for observational studies investigating the association between vitamin D status and the risk of gestational diabetes. Odds ratios (OR) or risk ratios (RR) from individual studies were pooled using the fixed and random effect models. Results: The meta-analysis of 29 observational studies included 28,982 participants, of which 4,634 were diagnosed with gestational diabetes, and showed that maternal vitamin D insufficiency was associated with a significantly increased risk of gestational diabetes by 39% (pooled OR = 1.39, 95%CI = 1.20-1.60) with moderate heterogeneity (I2 = 50.2%; P = 0.001). Moreover, the 25(OH)D level was significantly lower in gestational diabetes cases than in controls with a pooled effect of -4.79 nmol/L (95% CI = -6.43, -3.15). Significant heterogeneity was also detected (I2 = 65.0%, P &#x3c; 0.001). Further subgroup analysis indicated that this association was also evident in most subpopulations. Conclusion: This meta-analysis indicated a significant association between vitamin D insufficiency and increased risk of gestational diabetes. Further well-designed large-scale clinical trials are essential to verify this association

Association of maternal folate status in the second trimester of pregnancy with the risk of gestational diabetes mellitus

Interest in the high folate status of pregnant women has increased due to its role in the prevention of neural tube defects (NTDs). The effect of increased red blood cell (RBC) folate status during the second trimester of pregnancy on gestational diabetes mellitus (GDM) remains unclear. We measured RBC folate concentrations by competitive protein‐binding assay and obtained clinical information from electronic medical records. Logistic regression analysis was used to explore the associations of RBC folate concentrations with risks of gestational diabetes mellitus (GDM). We further assessed the potential nonlinear relations between continuous log‐transformed RBC folate concentrations and GDM risk by using the restricted cubic splines. We observed high RBC folate concentrations in GDM patients compared to control group [median (interquartile range, IQR), GDM vs. controls: 1,554.03 (1,240.54–1,949.99) vs. 1,478.83 (1,124.60–1,865.71) nmol/L, p = .001]. Notably, high folate concentrations were significantly associated with an increased risk of GDM [RR per 1‐SD increase: 1.16 (1.03, 1.30), p = .012] after adjustment for maternal age, parity, and body mass index (BMI) at enrollment. In the restricted cubic spline model, a test of the null hypothesis of the linear relationship was rejected (p = .001). Our study firstly showed that maternal RBC folate concentrations during the second trimester of pregnancy increase the risk of GDM in a Chinese population. Further randomized clinical trials (RCTs) are warranted to confirm the adverse effect